Skip to content

Add support for OAR Scheduler #713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Conversation

ychiat35
Copy link

@ychiat35 ychiat35 commented Oct 16, 2023

Types of changes

  • New feature (non-breaking change which adds functionality)

Summary

The Oar scheduler is widely used in France, including mesocentre supercomputers (e.g., GRICAD), INRIA supercomputers, Grid5000 testbed and other platforms.

This PR adds support for the OAR Scheduler, following the implementation made for Slurm.

Checklist

  • I have added tests to cover my changes (if necessary)
  • I have updated documentation (if necessary)

@codecov
Copy link

codecov bot commented Oct 18, 2023

Codecov Report

Attention: Patch coverage is 20.85561% with 148 lines in your changes missing coverage. Please review.

Project coverage is 88.02%. Comparing base (058d793) to head (5d15467).

Files with missing lines Patch % Lines
pydra/workers/oar.py 19.85% 108 Missing and 1 partial ⚠️
pydra/workers/tests/test_worker.py 22.91% 37 Missing ⚠️
pydra/conftest.py 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #713      +/-   ##
==========================================
- Coverage   88.76%   88.02%   -0.74%     
==========================================
  Files          84       85       +1     
  Lines       17890    18077     +187     
  Branches     3483     3528      +45     
==========================================
+ Hits        15880    15913      +33     
- Misses       1640     1792     +152     
- Partials      370      372       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@djarecka
Copy link
Collaborator

djarecka commented Nov 3, 2023

@ychiat35 - thank you for adding this worker. Would it be possible to test it within a container?

@ychiat35 ychiat35 force-pushed the feature/oar branch 9 times, most recently from ea8a805 to f7d9e79 Compare November 14, 2023 16:23
@ychiat35 ychiat35 force-pushed the feature/oar branch 3 times, most recently from 6856bfb to e54491c Compare November 21, 2023 16:07
@ychiat35
Copy link
Author

@ychiat35 - thank you for adding this worker. Would it be possible to test it within a container?

Hi, I'm trying to use oar-docker to do that. It is still WIP (I have to find more time doing that) but I noticed that the oar-docker solution requires cgroup v1, so I have to run on ubuntu-20.04 but not ubuntu-latest. Is this a problem for you?

@tclose
Copy link
Contributor

tclose commented Apr 1, 2025

Did you ever get this working? Should I try to port it to the new structure

@tclose tclose added the to consider suggesting changes that require more discussion label Apr 2, 2025
@github-project-automation github-project-automation bot moved this to In progress in Pydra Roadmap Apr 29, 2025
@tclose tclose removed the to consider suggesting changes that require more discussion label Apr 29, 2025
@ychiat35 ychiat35 marked this pull request as draft April 29, 2025 09:32
@tclose
Copy link
Contributor

tclose commented May 29, 2025

Thanks a lot for contributing this @ychiat35 !

The docs failure is an issue I'm running into in all my GH Actions that need MRtrix so should hopefully sort that out with the MRtrix conda pkg maintainers soon.

So the code looks good on a first pass but we need to add some way to install/setup a OAR scheduler the github action so we can test it.

There is also the question of whether this is better maintained in a separate plugin rather in the main code base or not. Do you have any thoughts on this @effigies @djarecka @satra ? I would probably lean towards a separate plugin due to the effort required to setup a test environment for it (like we have proposed for the SGE worker #506). I have created a template repo for such plugins here, https://github.com/nipype/pydra-workers-template

@satra
Copy link
Contributor

satra commented May 29, 2025

a separate repo makes it easier for long term maintenance or abandonment.

@tclose
Copy link
Contributor

tclose commented May 29, 2025

@ychiat35 is it possible to set this up as a separate plugin/repo using the https://github.com/nipype/pydra-workers-template template? If you need a hand with this at all let me know

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

5 participants